Projective Nonnegative Matrix Factorization: Sparseness, Orthogonality, and Clustering

Abstract In image compression and feature extraction, linear expansions are standardly used. It was pointed out by Lee and Seung that the positivity or non-negativity of a linear expansion is a very powerful constraint, that seems to lead to sparse representations for the images. Their technique, called Non-negative Matrix Factorization (NMF), was shown to be useful in approximating high dimensional data where the data are comprised of non-negative components. We have earlier proposed a new variant of the NMF method, called Projective Nonnegative Matrix Factorization, for learning spatially localized, sparse, part-based subspace representations of visual patterns. The algorithm is based on positively constrained projections and is related both to NMF and to the conventional SVD or PCA decomposition. In this paper we show that PNMF is intimately related to ”soft” k-means clustering and is able to outperform NMF in document classification tasks. The reason is that PNMF derives bases which are somewhat better for a localized representation than NMF, more orthogonal, and produce considerably more sparse representations.